QFDs and BLACHOLE 
Technology behind GCHQ/INOC 
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BLACK HOLE 



• Flat data store - not a database 



12 % 



~ 1,100,000,000,000 events since 6 th August 2007 

- 53.3TB compressed 

- 217TB uncompressed 

- 47% of data is from last 3 months 




19 % 



~10 Billion events per day 

Cheap system: 

- £1 k per TB of storage 

- £20k per lOGbps probe 

- ~£400k for lOxIOGbps prototype 




Websearch 
SMTP 
Webmail 
CNE Server Log 
Social networking 
Instant Messenger 
Routing Protocols 
Anonymiser 
Other 
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Questions - INOC Today 



Who, Where, When, What? - Online presence (TDI) 



MUTANT BROTH 



Who, which website(s)? - Online browsing activity 



KARMA POLICE 

Who, which fora?- Bulletin board usage 





INFINITE MONKEYS 



Where? - Google maps/earth usage 
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Questions - INOC Today 



Who, Where, When, What? - Online presence (TDI) 



MUTANT BROTH 



Who, which website(s)? - Online browsing activity 



KARMA POLICE 

Who, which fora?- Bulletin board usage 





INFINITE MONKEYS 



Where? - Google maps/earth usage 
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Questions - INOC Today 



Too many specific questions... Too much for analysts to learn. 
Need analyst systems to be simple and intuitive. 





Let analysts find & simply understand their targets easily... 



drill in to systems only when they need to 
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AR Prototypes - EPR Stats - Feb 2008 - Jan 2009 
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Number of users 
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